Information processing apparatus, information processing method, and computer program product

ABSTRACT

An information processing apparatus according to an embodiment includes one or more processors configured to: acquire an image and three-dimensional point information on a three-dimensional point of acquired surroundings of the image; assign an attribute to each region in the image; set, to the three-dimensional point, the attribute assigned to the corresponding region in the image; and generate an obstacle map based on the three-dimensional point information and the attribute that is set to the three-dimensional point.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromJapanese Patent Application No. 2018-077541, filed on Apr. 13, 2018, theentire contents of which are incorporated herein by reference.

FIELD

Embodiments described herein relate generally to an informationprocessing apparatus, an information processing method, and a computerprogram product.

BACKGROUND

There has been known a technology for generating an obstacle mapregarding obstacles to traveling with a mobile body. There has been alsoknown a technology that generates an obstacle map that specifies, inaddition to impassable regions in three-dimensional shapes in realspace, the regions that are impassable in travel rules even though theregions may be passable in three-dimensional shapes. For example, therehas been known a technology of identifying an attribute and updating theattribute for each frame on each voxel constituting a three-dimensionalgrid map generated from the stereo parallax.

However, in the conventional technology, because the identification ofthe attribute and the update of the attribute are performed on eachvoxel of the three-dimensional grid map for each frame, a processingload in generating the obstacle map may increase.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating a configuration of a mobile body;

FIG. 2 is a block diagram of the mobile body;

FIG. 3 is a schematic diagram of a captured image;

FIG. 4 is a schematic diagram of an attribute image;

FIG. 5 is a schematic diagram of the attribute image;

FIG. 6 is a diagram illustrating an orthogonal coordinate system;

FIG. 7 is a top view of the attribute image;

FIGS. 8A and 8B are schematic diagrams illustrating impassablenessprobabilities;

FIG. 9 is a schematic diagram illustrating the impassablenessprobabilities;

FIGS. 10A and 10B are schematic diagrams illustrating the impassablenessprobabilities;

FIGS. 11A, 11B, and 11C are schematic diagrams illustrating theimpassableness probabilities;

FIGS. 12A, 12B, and 12C are schematic diagrams illustrating theimpassableness probabilities;

FIG. 13 is a diagram illustrating an orthogonal coordinate system;

FIG. 14 is a schematic diagram of an obstacle map;

FIG. 15 is an explanatory diagram of time-series integration;

FIG. 16 is a schematic diagram of a display screen;

FIG. 17 is a flowchart of information processing;

FIG. 18 is a diagram illustrating a configuration of a mobile body; and

FIG. 19 is a hardware configuration diagram.

DETAILED DESCRIPTION

An information processing apparatus according to an embodiment includesone or more processors configured to: acquire an image andthree-dimensional point information on a three-dimensional point ofacquired surroundings of the image; assign an attribute to each regionin the image; set, to the three-dimensional point, the attributeassigned to the corresponding region in the image; and generate anobstacle map based on the three-dimensional point information and theattribute that is set to the three-dimensional point.

With reference to the accompanying drawings, an information processingapparatus, an information processing method, and a computer programproduct will be described in detail.

FIG. 1 is a diagram illustrating one example of a mobile body 10according to an embodiment.

The mobile body 10 includes an information processing apparatus 20, anoutput unit 10A, an outside sensor 10B, an inner sensor 10C, a powercontroller 10G, and a power unit 10H.

The information processing apparatus 20 is a dedicated orgeneral-purpose computer, for example. In the present embodiment, thecase where the information processing apparatus 20 is installed on themobile body 10 will be described as one example.

The mobile body 10 is an object that is movable. Examples of the mobilebody 10 include a vehicle, a dolly, a flying object (a manned aircraft,an unmanned aircraft (for example, an unmanned aerial vehicle (UAV) anda drone)), a robot, or the like. Examples of the mobile body 10 includea mobile body that travels via drive operation by a human and a mobilebody that is capable of autonomously traveling (autonomous travel)without going through the drive operation by a human. In the presentembodiment, the case where the mobile body 12 is a vehicle will bedescribed as one example. Examples of the vehicle include a two-wheeledvehicle, a three-wheeled vehicle, a four-wheeled vehicle, and the like.In the present embodiment, the case where the vehicle is a four-wheeledvehicle capable of autonomous travel will be described as one example.

The information processing apparatus 20 is not limited to the form ofbeing installed on the mobile body 10. The information processingapparatus 20 may be installed on a stationary object. The stationaryobject is an object fixed to the ground. The stationary object is anobject that is immovable and an object that is in a state of standingstill with respect to the ground. Examples of the stationary objectinclude a traffic signal, a parked vehicle, a road sign, or the like. Inaddition, the information processing apparatus 20 may be installed on acloud server that executes processing on a cloud.

The power unit 10H is a drive device installed on the mobile body 10.Examples of the power unit 10H include an engine, a motor, a wheel, orthe like.

The power controller 10G controls the power unit 10H. By the control ofthe power controller 10G, the power unit 10H drives. For example, thepower controller 10G controls the power unit 10H in order toautonomously drive the mobile body 10 on the basis of informationobtained from the outside sensor 10B and the inner sensor 10C, and anobstacle map generated by processing which will be described later. Bythe control of the power unit 10H, controlled are an acceleration amountof the mobile body 10, a brake amount, a steering angle, and the like.For example, the power controller 10G controls the vehicle so as tomaintain the lane currently traveling while avoiding an object such asan obstacle and to maintain an inter-vehicular distance of a certaindistance or more with a vehicle ahead.

The output unit 10A outputs various information. In the presentembodiment, the output unit 10A outputs information indicating anobstacle map generated by the information processing apparatus 20. Thedetail of the obstacle map will be described later.

The output unit 10A includes, for example, a communication function oftransmitting the information indicating the obstacle map, a displayfunction of displaying the obstacle map, a sound output function ofoutputting sound indicating the obstacle map, and the like. For example,the output unit 10A includes at least one of a communicator 10D, adisplay 10E, and a speaker 10F. In the present embodiment, theconfiguration in which the output unit 10A includes the communicator10D, the display 10E, and the speaker 10F will be described as oneexample.

The communicator 10D transmits the information indicating the obstaclemap to other devices. For example, the communicator 10D transmits theinformation indicating the obstacle map to the other devices via a knowncommunications line. The display 10E displays the information indicatingthe obstacle map. Examples of the display 10E include a known organicelectroluminescent (EL) display, a liquid crystal display (LCD), aprojector, a light, or the like. The speaker 10F outputs the soundindicating the obstacle map.

The outside sensor 10B is a sensor that recognizes the outside aroundthe periphery of the mobile body 10. The outside sensor 10B may beinstalled on the mobile body 10 or may be installed on the outside ofthe mobile body 10. Examples of the outside of the mobile body 10include another mobile body, an external device, or the like.

The periphery of the mobile body 10 is a region within a rangepredetermined from the mobile body 10. This range is a range in whichthe outside sensor 10B can observe. The range only needs to be set up inadvance.

The outside sensor 10B acquires observation information on the outside.The observation information is the information indicating an observationresult of the periphery of the position where the outside sensor 10B isplaced. In the present embodiment, the observation information is theinformation from which an image and three-dimensional point informationcan be derived. The observation information may be the informationindicating the image and the three-dimensional point information.

In the present embodiment, the outside sensor 10B includes aphotographing unit 10J. In the present embodiment, the photographingunit 10J acquires a captured image as the image. The image included inthe observation information is not limited to the captured image. Thecaptured image may be a three-dimensional image or may be atwo-dimensional image. In the present embodiment, the case where theimage included in the observation information is a two-dimensionalcaptured image will be described as one example.

The photographing unit 10J obtains a captured image that photographedthe periphery of the mobile body 10. The photographing unit 10J is aknown digital camera, for example. Photographing designates convertingan image of a subject that is formed by an optical system such as a lensinto an electric signal. The photographing unit 10J outputs thephotographed captured image to the information processing apparatus 20.

In the present embodiment, the outside sensor 10B further includes ameasurer 10K. The measurer 10K measures three-dimensional pointinformation on a three-dimensional point of acquired surroundings of thecaptured image.

The three-dimensional point represents each point that is individuallyobserved by the measurer 10K in the acquired surroundings of thecaptured image, that is, the photographed surroundings. For example, themeasurer 10K emits light to the periphery of the measurer 10K andreceives reflected light reflected at a reflecting point. Thisreflecting point is equivalent to the three-dimensional point. Aplurality of reflecting points may be used as one three-dimensionalpoint.

The three-dimensional point information is the information indicating athree-dimensional position in real space (three-dimensional space). Thethree-dimensional point information may be information indicative of thedistance from the measurer 10K to the three-dimensional point and of thedirection of the three-dimensional point with reference to the measurer10K. The distance and the direction can be represented by positioncoordinates indicating a relative position of the three-dimensionalpoint with reference to the measurer 10K, position coordinatesindicating an absolute position of the three-dimensional point, avector, or the like, for example. Specifically, the three-dimensionalpoint information is represented in polar coordinates and orthogonalcoordinates.

Examples of the measurer 10K include a stereo camera, a single-lenscamera, a distance sensor (a millimeter-wave radar and a laser sensor),a sonar sensor that detects an object by sound waves, an ultra-sonicsensor, or the like. Examples of the laser sensor include atwo-dimensional laser imaging detection and ranging (LIDAR) sensorplaced in parallel with the horizontal plane, and a three-dimensionalLIDAR sensor.

When the measurer 10K is a stereo camera, the measurer 10K only needs tomeasure, by using parallax information between the cameras, thethree-dimensional point information on the three-dimensional point. Whenthe measurer 10K is a single-lens camera, the measurer 10K may measure,based on the changes in the captured image along with the movement ofthe mobile body 10 identified by using structure-from-motion or thelike, the three-dimensional point information on the three-dimensionalpoint.

When the measurer 10K is a laser sensor, the measurer 10K obtains theobservation information that includes the irradiation direction of thelaser, and the information on the reflected light of the laser at thethree-dimensional point. Examples of the information on the reflectedlight include the elapsed time from the irradiation of the laser to thereception of the reflected light, the intensity of the receivedreflected light, an attenuation rate of the intensity of the receivedreflected light to the intensity of the emitted laser, or the like. Inthis case, the measurer 10K only needs to measure the three-dimensionalpoint information, by deriving the three-dimensional point informationon the three-dimensional point by using the observation information anda known calculation measurement method.

The measurer 10K then outputs the three-dimensional point information onthe measured three-dimensional point to the information processingapparatus 20. The information processing apparatus 20 may derive thethree-dimensional point information on each three-dimensional point fromthe observation information. In the present embodiment, the case wherethe measurer 10K derives the three-dimensional point information on eachthree-dimensional point will be described as one example.

In the present embodiment, the case where the photographing unit 10J isplaced in a travel direction of the mobile body 10 as a photographingdirection will be described as one example. Furthermore, the case wherethe measurer 10K is placed in the travel direction of the mobile body 10as a measuring direction will be described as one example.

Because of this, in the present embodiment, the case where thephotographing unit 10J and the measurer 10K acquire, in the traveldirection (that is, the front) of the mobile body 10, the captured imageand the three-dimensional point information on each three-dimensionalpoint will be described as one example.

The inner sensor 10C is a sensor that observes the information on themobile body 10 itself. The inner sensor 10C acquires self-locationinformation indicating the position of the mobile body 10. As in theforegoing, in the present embodiment, the mobile body 10 is equippedwith the photographing unit 10J and the measurer 10K. Accordingly, it ispossible to say that the self-location information is the informationindicating the position of the mobile body 10 and also the positions ofthe photographing unit 10J and the measurer 10K.

Examples of the inner sensor 10C include an inertial measurement unit(IMU), a velocity sensor, a global positioning system (GPS), or thelike. The IMU obtains triaxial acceleration, triaxial angular velocity,and the like of the mobile body 10. The self-location information isassumed to be represented in world coordinates.

Next, an electrical configuration of the mobile body 10 will bedescribed in detail. FIG. 2 is a block diagram illustrating one exampleof the configuration of the mobile body 10.

The mobile body 10 includes the information processing apparatus 20, theoutput unit 10A, the outside sensor 10B, the inner sensor 10C, the powercontroller 10G, and the power unit 10H. As in the foregoing, the outputunit 10A includes the communicator 10D, the display 10E, and the speaker10F.

The information processing apparatus 20, the output unit 10A, theoutside sensor 10B, the inner sensor 10C, and the power controller 10Gare connected via a bus 101. The power unit 10H is connected to thepower controller 10G.

The information processing apparatus 20 includes a storage 20B and aprocessor 20A. That is, the output unit 10A, the outside sensor 10B, theinner sensor 10C, the power controller 10G, the processor 20A, and thestorage 20B are connected via the bus 10I.

At least one of the storage 20B, the output unit 10A (the communicator10D, the display 10E, and the speaker 10F), the outside sensor 10B, theinner sensor 10C, and the power controller 10G only needs to beconnected to the processor 20A via wiring or in a wireless manner.Furthermore, at least one of the storage 20B, the output unit 10A (thecommunicator 10D, the display 10E, and the speaker 10F), the outsidesensor 10B, the inner sensor 10C, and the power controller 10G may beconnected to the processor 20A via a network.

The storage 20B stores therein various data. Examples of the storage 20Binclude a semiconductor memory device such as a random access memory(RAM) and a flash memory, a hard disk, an optical disc, and the like.The storage 20B may be a storage device provided externally to theinformation processing apparatus 20. The storage 20B may be a storagemedium. Specifically, the storage medium may be a medium storing thereinor temporarily storing therein programs and various information bydownloading via a local area network (LAN), the Internet, and the like.The storage 20B may be composed of a plurality of storage media.

The processor 20A includes an acquirer 20C, an assignor 20D, a setter20E, a generator 20F, and an output controller 20G. The generator 20Fincludes a first generator 20H, a second generator 20I, and anintegrator 20J.

The processor 20A, the storage 20B, the acquirer 20C, the assignor 20D,the setter 20E, the generator 20F, the output controller 20G, the firstgenerator 20H, the second generator 20I, and the integrator 20J areimplemented by one or more processors, for example. For example, theabove-described various units are implemented by one or more processors.For example, the above-described various units may be implemented bymaking the processor such as a central processing unit (CPU) execute aprogram, that is, by software. The above-described various units may beimplemented by a processor such as a dedicated integrated circuit (IC),that is, by hardware. The above-described various units may beimplemented by using the software and the hardware in combination. Whena plurality of processors are used, each processor may implement one outof the various units or may implement two or more out of the variousunits.

The processor implements the above-described various units by readingout and executing the program stored in the storage 20B. In place ofstoring the program in the storage 20B, it may be configured such thatthe program is directly incorporated in the circuitry of the processor.In this case, the processor implements the above-described various unitsby reading out and executing the program incorporated in the circuitry.

The acquirer 20C acquires the captured image, and the three-dimensionalpoint information on the three-dimensional point of the acquiredsurroundings (photographed surroundings) of the captured image.

In the present embodiment, the acquirer 20C acquires the captured imagefrom the photographing unit 10J. The acquirer 20C further acquires thethree-dimensional point information on each of a plurality ofthree-dimensional points from the measurer 10K.

In the present embodiment, the acquirer 20C acquires, each timephotographing is performed by the photographing unit 10J, a set of thecaptured image obtained by photographing, the self-location informationindicating the position of the mobile body 10 at the time ofphotographing, and the three-dimensional point information on each ofthe three-dimensional points of the photographed surroundings of acaptured image 30. That is, the acquirer 20C acquires a plurality ofrelevant sets in time series.

FIG. 3 is a diagram illustrating one example of the captured image 30.The captured image 30 illustrated in FIG. 3 is a captured image obtainedby photographing the front of the mobile body 10. In the captured image30, photographed are a roadway, sidewalks on the sides of the roadway,parked vehicles (other vehicles), and buildings. In the presentembodiment, by photographing by the photographing unit 10J fitted to themobile body 10, the acquirer 20C acquires in time series a capturedimage that photographed the range concerning the travel of the mobilebody 10 like the image illustrated in FIG. 3.

Referring back to FIG. 1, the description is continued. The assignor 20Dassigns an attribute to each region in the captured image 30. In moredetail, the assignor 20D determines the attribute for each of aplurality of regions obtained by dividing the captured image 30.Examples of the region in the captured image 30 include a pixel regionof each pixel constituting the captured image 30, a pixel regioncomposed of a plurality of adjacent pixel groups, and the like. Theassignor 20D then assigns the determined attribute to each of thoseregions.

The attribute indicates the characteristics and nature of each group forwhich the surroundings of the periphery of the mobile body 10 iscategorized to a plurality of groups in accordance with a predeterminedcondition. In the present embodiment, the attribute indicates beingpassable, being impassable, or a solid object.

The assignor 20D determines the attribute for each region in thecaptured image 30 by using a known method. For example, the assignor 20Ddetermines the attribute by using machine learning. For the attributedetermination using machine learning, a known method only needs to beused. For example, the assignor 20D determines the attribute for eachregion in the captured image 30 by using the method of J. Long, et. al,“Fully Convolutional Networks for Semantic Segmentation”, CVPR 2015; V.Badrinarayanan, et. al, “SegNet: A Deep Convolutional Encoder-DecoderArchitecture for Robust Semantic Pixel-Wise Labelling”, CVPR 2015; orthe like.

In this way, the assignor 20D assigns, to each region in the capturedimage 30, the attribute of the relevant region. For example, when theregion represents the pixel region of one pixel, the assignor 20Ddetermines the attribute for each pixel. Then, the assignor 20D assigns,to each region (pixel region) in the captured image 30, the attribute byspecifying the attribute determined for each pixel as a pixel value ofthe pixel.

FIG. 4 is a schematic diagram illustrating one example of an attributeimage 32 for which the attribute is assigned to each region E in thecaptured image 30. The assignor 20D generates the attribute image 32illustrated in FIG. 4, by assigning the attribute to each region E inthe captured image 30, for example.

In the present embodiment, the assignor 20D assigns, on the capturedimage 30, the attributes (passable and impassable) different from eachother to the roadway that is a passable region E2 and to the sidewalksthat are impassable regions E3. Although the sidewalk is a region thatthe mobile body 10 is able to pass through, it is the region wherepassing is prohibited by traffic rules. For such a region E3, theassignor 20D assigns the attribute indicative of being impassable. Theassignor 20D further assigns the attribute indicating a solid object toa region E1 indicating other vehicles and buildings included in thecaptured image 30.

The assignor 20D then outputs to the setter 20E the attribute image 32for which the attribute has been assigned to each region E in thecaptured image 30.

Referring back to FIG. 2, the description is continued. Next, the setter20E will be described. The setter 20E sets, to the three-dimensionalpoint, the attribute assigned to the corresponding region E in thecaptured image 30.

The setter 20E identifies the correspondence between thethree-dimensional point and the pixel of the captured image 30. That is,the setter 20E identifies, on each three-dimensional point indicated bythe three-dimensional point information acquired by the acquirer 20C,the pixel of the corresponding position out of the pixels constitutingthe captured image 30. Then, the setter 20E sets, to thethree-dimensional point, the attribute assigned to the region Eincluding the pixel of the identified corresponding position.

In more detail, the setter 20E converts, for each three-dimensionalpoint, the coordinates of the three-dimensional position indicated inthe three-dimensional point information into the coordinates (cameracoordinates) with the position of the photographing unit 10J, where thecaptured image 30 was photographed, as the origin. That is, thethree-dimensional position indicated in the three-dimensional pointinformation is the three-dimensional coordinates with the measurer 10Kas the origin. Accordingly, the setter 20E performs coordinatetransformation on the three-dimensional coordinates indicated by thethree-dimensional point information acquired from the measurer 10K, andconverts it into the camera coordinates with the photographing unit 10Jas the origin.

Specifically, a rotation matrix to the camera coordinates with thephotographing unit 10J at the time of photographing the captured image30 as the origin from the coordinates of a three-dimensional positionindicated by the three-dimensional point information is assumed to beR_(C). A translation vector is assumed to be t_(C). Then, the cameracoordinates (X_(C),Y_(C),Z_(C)) with the position of the photographingunit 10J at the time of photographing the captured image 30 as theorigin with respect to the coordinates (X_(in),Y_(in),Z_(in)) of thethree-dimensional point indicated by the three-dimensional pointinformation is expressed by the following Expression (1).

$\begin{matrix}{\begin{pmatrix}X_{C} \\Y_{C} \\Z_{C}\end{pmatrix} = {{R_{C}\begin{pmatrix}X_{in} \\Y_{in} \\Z_{in}\end{pmatrix}} + t_{C}}} & (1)\end{matrix}$

The rotation matrix R_(C) is a matrix of 3×3, and the translation vectort_(C) is a vector of three elements.

The setter 20E then converts the coordinates of the three-dimensionalposition indicated by the three-dimensional point information on thethree-dimensional point into the camera coordinates by using theabove-described Expression (1).

As in the foregoing, the attribute image 32 is the image for which theattributes have been assigned to the regions E (for example, pixelregion) in the captured image (see FIG. 4). Thus, the setter 20E thencalculates, on the three-dimensional point indicated by the coordinates(M=(X_(C),Y_(C),Z_(C))) of the three-dimensional position after thecoordinate transformation, the coordinates (m=(x_(C),y_(C))) of thepixel of the corresponding position in the attribute image 32 by usingthe following Expression (2).m′=A _(C) ·M′  (2)

In Expression (2), m′ represents the homogeneous coordinates of thecoordinates m=(x_(C),y_(C)). M′ represents the homogeneous coordinatesof the coordinates M=(X_(C),Y_(C),Z_(C)). A_(C) represents internalparameters of the photographing unit 10J.

The internal parameters of the photographing unit 10J represent thecharacteristics of the photographing unit 10J. Examples of the internalparameters of the photographing unit 10J include a focal length of lensof the photographing unit 10J, the position or the position coordinatesof the image center of the element with respect to the optical axis ofthe lens, and the like.

There may be the case where a photographing device is a stereo cameraand the like and the measurer 10K measures the three-dimensional pointinformation on the three-dimensional point on the basis of a stereoimage obtained by photographing. In this case, in the process ofmeasuring the three-dimensional point information by the measurer 10K,the coordinates (x_(C),y_(C)) in the captured image 30 corresponding tothe three-dimensional coordinates (X_(C),Y_(C),Z_(C)) is known. Thus, inthis case, the calculation of Expression (1) is unnecessary.

It is assumed that the internal parameters A_(C) of the photographingunit 10J, the rotation matrix R_(C) that is an external parameter, andthe translation vector t_(C) have been acquired in advance by priorcalibration.

The setter 20E then sets, to each three-dimensional point, the attributecalculated from the coordinates (M=(X_(C),Y_(C),Z_(C))) after thecoordinate translation and assigned to the region E of the correspondingcoordinates (m=(x_(C),y_(C))) in the attribute image 32. By thisprocessing, the setter 20E sets, to the three-dimensional point, theattribute assigned to the corresponding region E in the captured image30.

FIG. 5 is an attribute image 34 in which the three-dimensional point Pof the coordinates (M=(X_(C),Y_(C),Z_(C))) after coordinate translationis placed, in the attribute image 32, at the pixel position indicated bythe corresponding coordinates (m=(x_(C),y_(C))) calculated by theabove-described Expression (2).

In FIG. 5, the three-dimensional point P placed in the region E1 towhich the attribute indicating a solid object is assigned is indicatedas a three-dimensional point P1 of a circle mark. That is, the attribute“solid object” is set to the three-dimensional point P1. Furthermore, inFIG. 5, the three-dimensional point P placed in the region E2 to whichthe attribute indicative of being passable is assigned is indicated as athree-dimensional point P2 of a star mark. That is, the attribute“passable” is set to the three-dimensional point P2. In FIG. 5, thethree-dimensional point P placed in the region E3 to which the attributeindicative of being impassable is assigned is indicated as athree-dimensional point P3 of a rhombus mark. That is, the attribute“impassable” is set to the three-dimensional point P3.

The setter 20E then outputs to the generator 20F the three-dimensionalpoint information on the three-dimensional points P (for example, thethree-dimensional point P1 to the three-dimensional point P3) and theattributes set to the respective three-dimensional points P.

Referring back to FIG. 2, the description is continued. Next, thegenerator 20F will be described.

The generator 20F generates an obstacle map on the basis of thethree-dimensional point information on the three-dimensional point P andthe attribute set to the three-dimensional point P.

In the present embodiment, the generator 20F generates the obstacle mapindicating, for each sectional region, the degree of being passable orimpassable.

The sectional region indicates each region for which the space in theperiphery of the photographing unit 10J (the mobile body 10) is dividedinto a plurality of sectional regions in a lattice form, along a polarcoordinate system with the photographing unit 10J (the mobile body 10)as the origin. That is, in the present embodiment, the position of thephotographing unit 10J and the mobile body 10 is used as a referenceposition.

The size of the sectional region is greater than or equal to the size ofthe three-dimensional point P measured by the measurer 10K. That is, thesize of the sectional region may be the same as the size of thethree-dimensional point P. Furthermore, the size of the sectional regionmay be the size capable of including a plurality of three-dimensionalpoints P. That is, the size of the sectional region may be the same asthe size corresponding to the sensor resolution that is the maximumdensity of the three-dimensional point that can be measured by themeasurer 10K, or may be the size greater than the size corresponding tothe sensor resolution. The maximum size that the sectional region canassume only needs to be adjusted as appropriate.

In the present embodiment, the case where the size of the sectionalregion is greater than or equal to the size of the three-dimensionalpoint P will be described. Thus, in the present embodiment, the casewhere, in one sectional region, one or more three-dimensional points Pare included will be described as one example.

The degree of being passable or impassable is represented byprobability, for example. The degree of being passable or impassable isnot limited to the form of representing by the probability. In thepresent embodiment, the form in which the generator 20F represents thedegree of being passable or impassable by the probability will bedescribed as one example.

Furthermore, in the present embodiment, the case where the generator 20Fgenerates an obstacle map in which the impassableness probability isspecified for each sectional region will be described as one example.The impassableness probability indicates the probability of beingimpossible for the mobile body 10 to travel. The impassablenessprobability means that the probability of being impassable is higher asthe value is higher.

In more detail, the generator 20F first identifies the positionalinformation on the photographing unit 10J at the photographed time ofthe captured image 30. The positional information only needs to beidentified from the self-location information acquired at thephotographed time of the relevant captured image 30. As in theforegoing, the positional information on the photographing unit 10J isrepresented by the world coordinates.

Then, the generator 20F divides the space of the periphery of thephotographing unit 10J (the mobile body 10) into a plurality ofsectional regions in a lattice form, along the polar coordinate systemwith the photographing unit 10J (the mobile body 10) as the origin.

FIG. 6 is a diagram illustrating in an orthogonal coordinate system thesectional regions for which the space in the periphery of thephotographing unit 10J is divided, along the polar coordinate systemwith the photographing unit 10J as the origin. In FIG. 6, illustrated isan example in which the range of 180° of the front of the photographingunit 10J is divided into 18 angular directions by sectioning for each10°.

Then, the generator 20F converts, on the three-dimensional point P towhich the attribute has been set, the coordinates of thethree-dimensional point P from the orthogonal coordinates (x,z) into thepolar coordinates (r,θ), by using the following Expression (3) andExpression (4).r=√{square root over (x ² +z ²)}  (3)θ=a tan(z/x)  (4)

Then, as illustrated in FIG. 6, the generator 20F calculates, for eachangular direction of the polar coordinates with the photographing unit10J as the origin, the impassableness probability corresponding to theattribute set to the three-dimensional point P.

In the present embodiment, the case where the generator 20F specifies,for each angular direction, the impassableness probability correspondingto the attribute set to the three-dimensional point P will be describedas one example. In other words, in the present embodiment, the generator20F generates the obstacle map by specifying, for each angulardirection, the impassableness probability of each sectional regionarrayed along the angular direction.

The impassableness probability corresponding to the attribute only needsto be defined in advance.

In the present embodiment, the generator 20F sets a first impassablenessprobability, as the impassableness probability, to the sectional regioncorresponding to the three-dimensional point P1 to which the attributeindicating a solid object has been set. In the present embodiment, asthe first impassableness probability, “1.0” indicating the maximumimpassableness probability is set. This is to indicate that thesectional region corresponding to the three-dimensional point P1 towhich the attribute indicating a solid object has been set is impossibleto travel due to the presence of an obstacle.

Furthermore, the generator 20F specifies a second impassablenessprobability, as the impassableness probability, to the sectional regioncorresponding to the three-dimensional point P2 to which the attributeindicative of being passable has been set. The second impassablenessprobability is a value lower than that of the first impassablenessprobability. In the present embodiment, the case where “0.0” indicatingthe least value is set as the second impassableness probability will bedescribed. This is to indicate that the sectional region correspondingto the three-dimensional point P2 to which the attribute indicative ofbeing passable has been set is possible to travel due to the absence ofan obstacle.

The generator 20F further sets the first impassableness probability“1.0”, as the impassableness probability, to the sectional regioncorresponding to the three-dimensional point P3 to which the attributeindicative of being impassable has been set. This is to indicate thatthe sectional region corresponding to the three-dimensional point P3 towhich the attribute indicative of being impassable has been set has highprobability of being impassable due to the regulations of the trafficrules and the like.

In addition, the generator 20F specifies an intermediate probability, asthe impassableness probability, to the sectional region that is arrangedfarther away from the photographing unit 10J than, in the angulardirection with the position of the photographing unit 10J as the origin,the sectional region arranged at the position closest to thephotographing unit 10J and corresponding to the three-dimensional pointP1 to which the attribute indicating a solid object has been set. Theintermediate probability is an intermediate value between the firstimpassableness probability (1.0) and the second impassablenessprobability (0.0). In the present embodiment, the case where “0.5” isset as the intermediate probability will be described as one example.This is to indicate that the state of the relevant sectional region isunknown.

Then, the generator 20F specifies the impassableness probabilities, tothe sectional regions to which no impassableness probability has beenspecified in the above-described processing, on the basis of thesectional regions to which the impassableness probabilities (the firstimpassableness probability, the second impassableness probability, andthe intermediate probability) have been specified, by a method whichwill be described later. By the foregoing processing, the generator 20Fgenerates the obstacle map.

The generation processing of the obstacle map will be described indetail. FIG. 7 is a diagram illustrating the photographing surroundingsof the attribute image 34 illustrated in FIG. 5 as a top viewcorresponding to a bird's-eye view. As illustrated in FIG. 7, thegenerator 20F calculates, for each angular direction, the impassablenessprobability corresponding to the attribute set to the three-dimensionalpoint P.

In FIG. 7, as one example, three angular directions (the angulardirection A to the angular direction C) with the photographing unit 10J(the mobile body 10) as the origin are illustrated. In FIG. 7, it isassumed that, in the angular direction A, a building is present at theclosest position as viewed from the photographing unit 10J and islocated at the distance r₀ from the photographing unit 10J. Furthermore,it is assumed that, in the angular direction B, a vehicle is present atthe closest position as viewed from the photographing unit 10J and islocated at the distance r₀ from the photographing unit 10J. It isfurther assumed that, in the angular direction C, there are no solidobjects such as vehicles and buildings present.

Then, the generator 20F specifies, for each angular direction, theimpassableness probability corresponding to the attribute set to thethree-dimensional point P.

Referring back to FIG. 2, the description is continued. In the presentembodiment, the generator 20F includes the first generator 20H, thesecond generator 20I, and the integrator 20J.

First, the first generator 20H will be described. The first generator20H generates a first obstacle map in which the impassablenessprobability is specified to the sectional region corresponding to thethree-dimensional point P1 to which the attribute indicating a solidobject has been set. In more detail, the first generator 20H generates afirst obstacle map 40 by specifying, in the angular direction includingthe sectional region corresponding to the three-dimensional point P1 towhich the attribute indicating a solid object has been set, theimpassableness probability to each sectional region arrayed along therelevant angular direction.

First, the first generator 20H identifies, out of the angulardirections, an angular direction including the sectional regioncorresponding to the three-dimensional point P1 to which the attributeindicating a solid object has been set. In the case of the exampleillustrated in FIG. 7, the first generator 20H identifies the angulardirection A and the angular direction B.

Then, the first generator 20H specifies the first impassablenessprobability (1.0), as the impassableness probability, to the sectionalregions corresponding to the three-dimensional point P1 to which theattribute indicating a solid object has been set.

Next, the first generator 20H specifies the second impassablenessprobability (0.0), as the impassableness probability, to other sectionalregions within the range from the position of the photographing unit 10J(the mobile body 10) that is the reference position to the sectionalregion corresponding to the relevant three-dimensional point P1. This isbecause, in the sectional regions located closer to the photographingunit 10J than the solid object located at the closest position from thephotographing unit 10J (the mobile body 10), the probability of beingpassable is high as the obstacle is not present.

Moreover, the first generator 20H specifies the intermediate probability(0.5), as the impassableness probability, to other sectional regionswithin the range farther away from the photographing unit 10J (themobile body 10) that is the reference position than the sectional regioncorresponding to the three-dimensional point P1 to which the attributeindicating a solid object has been set. This is because, in thesectional regions located farther from the photographing unit 10J thanthe solid object located at the closest position from the photographingunit 10J (the mobile body 10), the state is unknown due to the shieldingby the solid object.

FIGS. 8A and 8B are schematic diagrams illustrating the impassablenessprobabilities that the first generator 20H specified. FIG. 8A is aschematic diagram illustrating by extracting a plurality of sectionalregions arrayed along the angular direction A and the angular directionB in FIG. 7. In other words, FIG. 8A is a schematic diagramillustrating, in the first obstacle map 40, the impassablenessprobability specified to each of a plurality of sectional regions 40Aarrayed along a specific angular direction.

As illustrated in FIG. 8A, the first generator 20H specifies the firstimpassableness probability “1.0” to the sectional region correspondingto the three-dimensional point P1 located at the distance r₀ from thephotographing unit 10J and to which the attribute indicating a solidobject has been set. Furthermore, the first generator 20H specifies thesecond impassableness probability “0.0” to the sectional regions thatare present in the range between the sectional region located at thedistance r₀ and the photographing unit 10J (the mobile body 10). Thefirst generator 20H further specifies the intermediate probability “0.5”to the sectional regions that are present in the range farther from thephotographing unit 10J (the mobile body 10) than the sectional regionlocated at the distance r₀.

As just described, the first generator 20H generates the first obstaclemap 40 by specifying, for each angular direction, out of the respectiveangular directions, including the sectional region corresponding to thethree-dimensional point P1 to which the attribute indicating a solidobject has been set, the impassableness probability to each sectionalregion arrayed along the relevant angular direction.

There may be the case where another solid object is present at theposition farther away from the photographing unit 10J than a solidobject that is present at the closest position (the position of thedistance r₀) as viewed from the photographing unit 10J. In this case, asillustrated in FIG. 8B, the first generator 20H only needs to furtherspecify, after specifying the impassableness probabilities in the samemanner as that with FIG. 8A, the first impassableness probability “1.0”to the sectional region arranged at the position farther from thephotographing unit 10J than the sectional region at the position of thedistance r₀ and corresponding to the three-dimensional point P1 to whichthe attribute indicating a solid object has been set.

There may be an angular direction that does not include thethree-dimensional point P1 to which the attribute indicating a solidobject has been set. For example, the angular direction C illustrated inFIG. 7 does not include the three-dimensional point P1 to which theattribute indicating a solid object has been set. In this case, thefirst generator 20H specifies “0.5” that is the intermediateprobability, to the sectional regions arrayed along the relevant angulardirection C.

FIG. 9 is a schematic diagram illustrating the impassablenessprobabilities that the first generator 20H specified. In more detail,FIG. 9 is a schematic diagram illustrating the impassablenessprobability specified to each of a plurality of sectional regions 40Carrayed along the angular direction C in FIG. 7. As illustrated in FIG.9, the first generator 20H specifies the intermediate probability “0.5”to the sectional regions arrayed along the angular direction C that doesnot include the three-dimensional point P1.

In this way, the first generator 20H generates the first obstacle map 40in which the impassableness probabilities are specified in theabove-described manner on the angular direction including the sectionalregion corresponding to the three-dimensional point P1 to which theattribute indicating a solid object has been set.

Next, the second generator 20I will be described. The second generator20I generates a second obstacle map in which the impassablenessprobabilities are specified to the sectional regions corresponding tothe three-dimensional points P (the three-dimensional point P2 and thethree-dimensional point P3) to which the attribute indicative of beingpassable or impassable has been set.

In more detail, the second generator 20I generates the second obstaclemap by specifying, in the angular direction including the sectionalregions corresponding to the three-dimensional points P (thethree-dimensional point P2 and the three-dimensional point P3) to whichthe attribute indicative of being passable or impassable has been set,the impassableness probability to each sectional region arrayed alongthe relevant angular direction.

The second generator 20I specifies the first impassableness probability“1.0”, out of the sectional regions arrayed along each angulardirection, to the sectional region corresponding to thethree-dimensional point P3 to which the attribute indicative of beingimpassable has been set.

Furthermore, the second generator 20I specifies the secondimpassableness probability “0.0”, out of the sectional regions arrayedalong each angular direction, to the sectional region corresponding tothe three-dimensional point P2 to which the attribute indicative ofbeing passable has been set.

Then, the second generator 20I sets, out of the sectional regionsarrayed along the identified angular direction, the intermediateprobability “0.5” to the sectional regions to which neither theattribute indicative of being passable nor the attribute indicative ofbeing impassable has been specified.

FIGS. 10A and 10B are schematic diagrams illustrating the impassablenessprobabilities that the second generator 20I specified. FIG. 10A is aschematic diagram illustrating, in a second obstacle map 42′, theimpassableness probability specified to each of a plurality of sectionalregions 42′E arrayed along a specific angular direction.

As illustrated in FIG. 10A, the second generator 20I specifies the firstimpassableness probability “1.0”, out of the sectional regions along therelevant angular direction, to the sectional region corresponding to thethree-dimensional point P3 to which the attribute indicative of beingimpassable has been set. Furthermore, the second generator 20I specifiesthe second impassableness probability “0.0”, out of the sectionalregions along the relevant angular direction, to the sectional regioncorresponding to the three-dimensional point P2 to which the attributeindicative of being passable has been set. In addition, the secondgenerator 20I specifies the intermediate probability “0.5”, out of thesectional regions along the relevant angular direction, to the sectionalregions to which no attribute indicative of being passable or impassablehas been specified.

Then, the second generator 20I specifies the impassablenessprobabilities based on the sectional regions to which the firstimpassableness probability “1.0” or the second impassablenessprobability “0.0” has been specified, to the sectional regions to whichthe intermediate probability “0.5” has been specified. That is, thesecond generator 20I complements the impassableness probabilities of thesectional regions to which the intermediate probability “0.5” has beenspecified, by using the impassableness probabilities of the peripheralsectional regions to which the first impassableness probability or thesecond impassableness probability has been specified.

Specifically, the second generator 20I complements the impassablenessprobabilities of the sectional regions to which the intermediateprobability “0.5” has been specified in FIG. 10A, by using theimpassableness probabilities of the peripheral sectional regions towhich the second impassableness probability “0.0” or the firstimpassableness probability “1.0” has been specified.

FIG. 10B is a schematic diagram illustrating the state in which thesectional regions to which the intermediate probability “0.5” has beenspecified in FIG. 10A are complemented (changed) by using theimpassableness probabilities that have been specified to the otherperipheral sectional regions.

As illustrated in FIG. 10A, the second generator 20I changes theimpassableness probability of the sectional regions to which theintermediate probability “0.5” has been specified to the impassablenessprobability specified to a pair of other sectional regions adjacent inthe angular direction.

Specifically, it is assumed that the sectional region to which theintermediate probability “0.5” has been specified is sandwiched, alongthe angular direction, by a pair of sectional regions to which thesecond impassableness probability “0.0” has been specified. In thiscase, the second generator 20I changes the impassableness probability ofthe relevant sectional region, to which the intermediate probability“0.5” has been specified, to the second impassableness probability “0.0”(see FIG. 10B).

Furthermore, it is assumed that the sectional region to which theintermediate probability “0.5” has been specified is sandwiched, alongthe angular direction, by a pair of sectional regions to which the firstimpassableness probability “1.0” has been specified. In this case, thesecond generator 20I changes the impassableness probability of therelevant sectional region, to which the intermediate probability “0.5”has been specified, to the first impassableness probability “1.0” (seeFIG. 10B).

It is further assumed that the sectional region to which theintermediate probability “0.5” has been specified is sandwiched, alongthe angular direction, by the sectional region to which the firstimpassableness probability “1.0” has been specified and the sectionalregion to which the second impassableness probability “0.0” has beenspecified. In this case, the second generator 20I specifies theimpassableness probability of the relevant sectional region, to whichthe intermediate probability “0.5” has been specified, to theintermediate probability “0.5” that is an intermediate value between thefirst impassableness probability and the second impassablenessprobability (see FIG. 10B).

In this way, the second generator 20I generates the second obstacle map42 by specifying, in the angular direction including the sectionalregions corresponding to the three-dimensional points P (thethree-dimensional point P2 and the three-dimensional point P3) to whichthe attribute indicative of being passable or impassable has been set,the impassableness probability to each sectional region arrayed alongthe relevant angular direction.

Thus, in the second obstacle map 42, the impassableness probabilityspecified to each of the sectional regions corresponding to therespective angular direction A to the angular direction C illustrated inFIG. 7 becomes the diagrams illustrated in FIGS. 11A to 11C. FIGS. 11Ato 11C are schematic diagrams illustrating the impassablenessprobabilities that the second generator 20I specified. FIG. 11A is aschematic diagram illustrating the impassableness probability specifiedto each of the sectional regions 42A arrayed along the angular directionA illustrated in FIG. 7. FIG. 11B is a schematic diagram illustratingthe impassableness probability specified to each of the sectionalregions 42B arrayed along the angular direction B illustrated in FIG. 7.FIG. 11C is a schematic diagram illustrating the impassablenessprobability specified to each of the sectional regions 42C arrayed alongthe angular direction C illustrated in FIG. 7.

The second generator 20I may, when complementing the sectional region towhich the intermediate probability “0.5” has been specified by using theimpassableness probabilities specified to the other peripheral sectionalregions, consider the attribute assigned to the region E (see FIG. 4)corresponding to the sectional region to which the intermediateprobability “0.5” has been specified.

This will be described specifically by using FIG. 10A. In FIG. 10A,assumed is the case of complementing the impassableness probability ofthe sectional region that is located at the second from the leftmostside and to which the intermediate probability “0.5” has been specified.In this case, as in the foregoing, the second generator 20I performscomplement processing by using the second impassableness probability“0.0” specified to each of the sectional regions corresponding to thethree-dimensional points P2 that are located at the leftmost side and atthe third from the left in FIG. 10A and to which the attributeindicative of being passable has been set.

It is assumed that the three-dimensional point P2 corresponding to thesectional region located at the leftmost side in FIG. 10A corresponds toa pixel of the pixel position (x1,y1) in the attribute image 32 (seeFIG. 4). Furthermore, it is assumed that, in FIG. 10A, thethree-dimensional point P2 corresponding to the sectional region locatedat the third from the leftmost side corresponds to a pixel of the pixelposition (x2,y2) in the attribute image 32 (see FIG. 4). It is furtherassumed that, in the attribute image 32, the range connecting the pixelposition (x1,y1) and the pixel position (x2,y2) is within the range ofthe sectional regions of complement target.

Then, assumed is the case where, in the attribute image 32, no attributeindicative of being impassable is assigned to the pixels (region E)located in the range connecting the pixel position (x1,y1) and the pixelposition (x2,y2). In this case, as illustrated in FIG. 10B, the secondgenerator 20I sets the second impassableness probability “0.0”corresponding to the attribute indicative of being passable to thesectional regions of complement target.

Meanwhile, assumed is the case where, in the attribute image 32, theattribute indicative of being impassable is assigned to the pixels(region E) located in the range connecting the pixel position (x1,y1)and the pixel position (x2,y2). In this case, the second generator 20Isets the intermediate probability “0.5” to the sectional regions ofcomplement target.

There may be the case where a plurality of three-dimensional points P towhich the attributes different from each other have been assignedcorrespond to one sectional region. For example, there may be the casewhere there is a sectional region corresponding to both thethree-dimensional point P2 to which the attribute indicative of beingpassable has been set and the three-dimensional point P3 to which theattribute indicative of being impassable has been set.

In this case, the second generator 20I only needs to specify, to therelevant section region, the first impassableness probability “1.0” thatis the impassable probability corresponding to the attribute indicativeof being impassable.

Next, the integrator 20J will be described. The integrator 20J generatesan obstacle map in which the first obstacle map 40 and the secondobstacle map 42 are integrated.

In more detail, the integrator 20J integrates the first obstacle map 40and the second obstacle map 42 on the basis of priority of theimpassableness probability.

The priority of the impassableness probability only needs to be set inadvance. In the present embodiment, the integrator 20J sets in advancethe priority for which the first impassableness probability “1.0” is thehighest, the second impassableness probability “0.0” is the secondhighest, and the intermediate probability “0.5” is the lowest.

Then, the integrator 20J specifies the impassableness probability ofhigher priority to each sectional region included in the obstacle map,out of the impassableness probability specified to the sectional regioncorresponding to the first obstacle map 40 and the impassablenessprobability specified to the sectional region corresponding to thesecond obstacle map 42. By this processing, the integrator 20J generatesthe obstacle map in which the first obstacle map 40 and the secondobstacle map 42 are integrated.

This will be described specifically by using FIGS. 12A to 12C. FIGS. 12Ato 12C are schematic diagrams illustrating, in an obstacle map 44, theimpassableness probability specified to each of the sectional regions(sectional region 44A to sectional region 44C) arrayed along therespective angular directions (angular direction A to angular directionC).

FIG. 12A is a schematic diagram illustrating, in the obstacle map 44,the impassableness probabilities specified to the sectional regions 44Aarrayed along the angular direction A. The integrator 20J integrates theimpassableness probabilities (see FIG. 8A) specified to the sectionalregions 40A arrayed along the angular direction A in the first obstaclemap 40 and the impassableness probabilities (see FIG. 11A) specified tothe sectional regions 42A arrayed along the angular direction A in thesecond obstacle map 42. By this integration, the integrator 20Jspecifies the impassableness probabilities illustrated in FIG. 12A onthe sectional regions 44A arrayed along the angular direction A.

FIG. 12B is a schematic diagram illustrating, in the obstacle map 44,the impassableness probabilities specified to the sectional regions 44Barrayed along the angular direction B. The integrator 20J integrates theimpassableness probabilities (see FIG. 8A) specified to the sectionalregions 40A arrayed along the angular direction B in the first obstaclemap 40 and the impassableness probabilities (see FIG. 11B) specified tothe sectional regions 42B arrayed along the angular direction B in thesecond obstacle map 42. By this integration, the integrator 20Jspecifies the impassableness probabilities illustrated in FIG. 12B onthe sectional regions 44B arrayed along the angular direction B.

FIG. 12C is a schematic diagram illustrating, in the obstacle map 44,the impassableness probabilities specified to the sectional regions 44Carrayed along the angular direction C. The integrator 20J integrates theimpassableness probabilities (see FIG. 9) specified to the sectionalregions 40C arrayed along the angular direction C in the first obstaclemap 40 and the impassableness probabilities (see FIG. 11C) specified tothe sectional regions 42C arrayed along the angular direction C in thesecond obstacle map 42. By this integration, the integrator 20Jspecifies the impassableness probabilities illustrated in FIG. 12C onthe sectional regions 44C arrayed along the angular direction C.

By the above-described processing, the integrator 20J generates theobstacle map 44 for which the first obstacle map 40 and the secondobstacle map 42 are integrated.

Then, the integrator 20J converts the obstacle map 44 generated in thepolar coordinate system into an orthogonal coordinate system. FIG. 13 isa diagram illustrating in the orthogonal coordinate system the relationbetween the sectional regions in the polar coordinate system and thesectional regions in the orthogonal coordinate system. In FIG. 13, thesectional regions sectioned by solid lines are the sectional regionsdivided in a rectangular shape in the orthogonal coordinate system. InFIG. 13, the sectional regions sectioned by broken lines are thesectional regions that are divided in a rectangular shape in the polarcoordinate system and displayed in the orthogonal coordinate system.

The integrator 20J identifies, on each sectional region represented inthe orthogonal coordinate system, the sectional region of the nearestposition out of the sectional regions represented in the polarcoordinate system, by using the nearest neighbor method. Then, theintegrator 20J specifies, to the respective sectional regionsrepresented in the orthogonal coordinate system, the impassablenessprobability identified to each sectional region and specified to theclosest sectional region in the polar coordinate system.

Furthermore, the integrator 20J identifies, on each sectional regionrepresented in the orthogonal coordinate system, the sectional regionlocated in the closest vicinity out of a plurality of sectional regionsrepresented in the polar coordinate system. The integrator 20J thencomplements the impassableness probability of the identified sectionalregion by using the bilinear method. Then, the integrator 20J mayspecify, to the sectional region of a specifying target represented inthe orthogonal coordinate system, the impassableness probabilityobtained by the relevant complement.

The foregoing methods are one example of coordinate transformation fromthe polar coordinate system into the orthogonal coordinate system, andthe embodiment is not limited to these methods.

By the above-described processing, the integrator 20J, that is, thegenerator 20F generates the obstacle map 44 corresponding to thebird's-eye view.

FIG. 14 is a diagram illustrating one example of the obstacle map 44. Bythe above-described processing, the generator 20F generates the obstaclemap 44. In FIG. 14, the sectional regions represented in white are thesectional regions to which the second impassableness probability “0.0”has been assigned. In FIG. 14, the sectional regions represented inblack are the sectional regions to which the first impassablenessprobability “1.0” has been assigned. In FIG. 14, the sectional regionsrepresented in gray are the sectional regions to which the intermediateprobability “0.5” has been assigned.

The generator 20F may time-serially integrate the generated obstacle map44 with the obstacle map 44 that has been generated in the past.

FIG. 15 is an explanatory diagram of time-series integration. FIG. 15illustrates a plurality of sectional regions obtained by dividing thespace of the periphery centering the mobile body 10 at each of time t−1and time t.

The region N_(t-1) that is the sectional region at the time t−1 and theregion N_(t) that is the sectional region at the time t differ in therelative position from the mobile body 10 at the respective time.However, these region N_(t-1) and region N_(t) indicate the sameposition in the world coordinate space.

Thus, the generator 20F calculates, from the self-location information,the amount of movement of the mobile body 10 between the time t and thetime t−1 that is the time immediately before that. Then, the generator20F obtains, based on the amount of movement of the mobile body 10, thesectional regions at the time t−1 that correspond to the respectivesectional regions at the time t. In the example in FIG. 15, the regionN_(t-1) is obtained as the sectional region at the time t−1 thatcorresponds to the region N_(t) at the time t. The generator 20F thenintegrates the impassableness probability specified to the region N_(t)and the impassableness probability specified to the region N_(t-1) inthe past. In this integration, it only needs to use Bayes' theoremexpressed in the following Expression (5), for example.

$\begin{matrix}{\frac{p\left( {m_{i}\text{❘}z_{1}\mspace{14mu}\ldots\mspace{14mu} z_{t}} \right)}{1 - {p\left( {m_{i}\text{❘}z_{1}\mspace{14mu}\ldots\mspace{20mu} z_{t}} \right)}} = {\frac{p\left( {m_{i}\text{❘}z_{t}} \right)}{1 - {p\left( {m_{i}\text{❘}z_{t}} \right)}} \cdot \frac{p\left( {m_{i}\text{❘}z_{1}\mspace{14mu}\ldots\mspace{20mu} z_{t - 1}} \right)}{1 - {p\left( {m_{i}\text{❘}z_{1\mspace{11mu}}\ldots\mspace{14mu} z_{t - 1}} \right)}}}} & (5)\end{matrix}$

In Expression (5), P(m_(i)|z_(t)) represents the impassablenessprobability that is based on the current positional information,p(m_(i)|z_(i), . . . , z_(t-1)) represents the impassablenessprobability that is based on the past positional information, andp(m_(i)|z₁, . . . , z_(i)) represents the impassableness probabilitythat is based on the positional information up until the present.

The generator 20F generates the obstacle map 44 for which theimpassableness probabilities specified to the respective sectionalregions are integrated in time series, and thus it is possible torobustly calculate the impassableness probabilities even when the sensorobserved a value including noise at any timing, for example.

Referring back to FIG. 2, the description is continued. The generator20F outputs the generated obstacle map 44 to the output controller 20G.

The output controller 20G outputs the information indicating theobstacle map 44. In the present embodiment, the output controller 20Goutputs the information indicating the obstacle map 44 to at least oneof the output unit 10A and the power controller 10G.

The output controller 20G displays the information indicating theobstacle map 44 on the display 10E. In the present embodiment, theoutput controller 20G displays a display screen including the obstaclemap 44 on the display 10E.

FIG. 16 is a schematic diagram illustrating one example of a displayscreen 60. The display screen 60 includes the obstacle map 44 generatedin the generator 20F. Because of this, the user can, by checking thedisplay screen 60, easily recognize the impassableness probabilities.The display screen 60 may be a screen for which, on the obstacle map 44,an image 60A indicating the mobile body 10 that is the own vehicle and aline 60B indicating a planned travel route or a recommended route of themobile body 10 are superimposed.

Referring back to FIG. 2, the description is continued. The outputcontroller 20G may further control the display 10E and the speaker 10Fso as to output the sound and the light indicating the obstacle map 44.Furthermore, the output controller 20G may transmit the informationindicating the obstacle map 44 to an external device via thecommunicator 10D.

The output controller 20G may output the information indicating theobstacle map 44 to the power controller 10G. That is, the outputcontroller 20G may output to the power controller 10G the informationindicating the impassableness probability specified to each sectionalregion included in the obstacle map 44.

In this case, the power controller 10G controls the power unit 10H inaccordance with the information indicating the obstacle map 44 receivedfrom the output controller 20G. For example, the power controller 10Gmay, in accordance with the impassableness probability specified to eachsectional region included in the obstacle map 44, generate a powercontrol signal for controlling the power unit 10H and control the powerunit 10H. The power control signal, in the power unit 10H, is a controlsignal for controlling a drive unit that performs the drive concerningthe travel of the mobile body 10. For example, the power controller 10Gcontrols the steering, the engine, or the like of the mobile body 10 sothat the mobile body 10 travels the regions of the real spacecorresponding to the sectional regions to which the secondimpassableness probability “0.0” has been specified in the obstacle map44.

Next, one example of a procedure of information processing that theinformation processing apparatus 20 performs will be described. FIG. 17is a flowchart illustrating one example of the procedure of informationprocessing.

First, the acquirer 20C acquires the captured image 30, and thethree-dimensional point information on the three-dimensional points P ofthe photographed surroundings of the captured image 30 (Step S100).

Then, the assignor 20D assigns the attribute to each region E in thecaptured image 30 acquired at Step S100 (Step S102).

The setter 20E then sets, to the three-dimensional points P indicated bythe three-dimensional point information acquired at Step S100, theattributes assigned to the corresponding regions E in the captured image30 (Step S104).

Then, the generator 20F generates the obstacle map 44 on the basis ofthe three-dimensional point information on the three-dimensional pointsP acquired at Step S100 and the attributes set to the three-dimensionalpoints P at Step S104 (Step S106).

The output controller 20G then outputs the information indicating theobstacle map 44 generated at Step S106 to the output unit 10A and thepower controller 10G (Step S108). Then, the present routine is ended.

As in the foregoing, the information processing apparatus 20 of thepresent embodiment includes the acquirer 20C, the assignor 20D, thesetter 20E, and the generator 20F. The acquirer 20C acquires thecaptured image 30 (an image) and the three-dimensional point informationon the three-dimensional points P of the acquired surroundings of thecaptured image 30. The assignor 20D assigns the attribute to each regionE in the captured image 30. The setter 20E sets, to thethree-dimensional points P, the attribute assigned to the correspondingregion E in the captured image 30. The generator 20F generates theobstacle map 44 on the basis of the three-dimensional point informationand the attributes set to the three-dimensional points P.

As in the foregoing, the information processing apparatus 20 of thepresent embodiment sets the attribute of the three-dimensional point Pon the basis of the attribute of the corresponding region E on thecaptured image 30, and generates the obstacle map 44 on the basis of theattribute set to the three-dimensional point P.

Because of this, the information processing apparatus 20 of the presentembodiment is able to generate the obstacle map 44 while suppressing theamount of calculations and the amount of memory used.

Consequently, the information processing apparatus 20 of the presentembodiment is able to achieve reducing a processing load.

Modifications

In the above-described embodiment, the form in which the outside sensor10B is provided externally to the information processing apparatus 20has been described as one example (see FIG. 2). The informationprocessing apparatus 20 may, however, be in a configuration thatincludes the outside sensor 10B.

FIG. 18 is a schematic diagram illustrating one example of theconfiguration of a mobile body 11 of a modification. The constituentelements having the same functions as those described in theabove-described embodiment are denoted by the same reference signs andthe detailed explanations thereof are omitted.

In the mobile body 11, the output unit 10A, the inner sensor 10C, thepower controller 10G, the power unit 10H, and an information processingapparatus 21 are connected via the bus 10I. The mobile body 11 is thesame as the mobile body 10 of the above-described embodiment except forthe point that the information processing apparatus 21 is included inlieu of the information processing apparatus 20.

The information processing apparatus 21 includes the storage 20B, theprocessor 20A, and the outside sensor 10B. The information processingapparatus 21 is the same as the information processing apparatus 20 ofthe above-described embodiment except for the point that the outsidesensor 10B is further included. The outside sensor 10B is the same asthat of the above-described embodiment.

As illustrated in FIG. 18, the information processing apparatus 21 maybe in a configuration that further includes, in addition to theprocessor 20A and the storage 20B, the outside sensor 10B (that is, thephotographing unit 10J and the measurer 10K).

The information processing apparatus 21 may be in a configuration thatfurther includes, in addition to the processor 20A and the storage 20B,at least one of the communicator 10D, the display 10E, the speaker 10F,the inner sensor 10C, and the power controller 10G.

Hardware Configuration

Next, one example of a hardware configuration of the informationprocessing apparatus 20 of the above-described embodiment and theinformation processing apparatus 21 of the modification will bedescribed. FIG. 19 is one example of a hardware configuration diagram ofthe information processing apparatus 20 and the information processingapparatus 21.

The information processing apparatus 20 and the information processingapparatus 21 include a control device such as a CPU 86; a storage devicesuch as a read only memory (ROM) 88, a random access memory (RAM) 90,and a hard disk drive (HDD) 92; an I/F unit 82 that is an interface withvarious devices; an output unit 80 that outputs various information suchas output information; an input unit 94 that receives operations by theuser; and a bus 96 that connects various units, and are in a hardwareconfiguration using an ordinary computer.

In the information processing apparatus 20 and the informationprocessing apparatus 21, the CPU 86 reads out and executes a programfrom the ROM 88 onto the RAM 90, thereby implementing theabove-described various units on the computer.

The program for executing each of the above-described processingexecuted in the information processing apparatus 20 and the informationprocessing apparatus 21 may be stored in the HDD 92. Furthermore, theprogram for executing each of the above-described processing executed inthe information processing apparatus 20 and the information processingapparatus 21 may be incorporated and provided in the ROM 88 in advance.

The program for executing the above-described processing executed in theinformation processing apparatus 20 and the information processingapparatus 21 may be stored in a computer-readable storage medium such asa CD-ROM, a CD-R, a memory card, a digital versatile disc (DVD), aflexible disk (FD), or the like in a file of an installable orexecutable format and provided as a computer program product. Theprogram for executing the above-described processing executed in theinformation processing apparatus 20 and the information processingapparatus 21 may be stored in a computer connected to a network such asthe Internet and be available for download via the network. The programfor executing the above-described processing executed in the informationprocessing apparatus 20 and the information processing apparatus 21 mayalso be provided or distributed via a network such as the Internet.

While certain embodiments have been described, these embodiments havebeen presented by way of example only, and are not intended to limit thescope of the inventions. Indeed, the novel embodiments described hereinmay be embodied in a variety of other forms; furthermore, variousomissions, substitutions and changes in the form of the embodimentsdescribed herein may be made without departing from the spirit of theinventions. The accompanying claims and their equivalents are intendedto cover such forms or modifications as would fall within the scope andspirit of the inventions.

What is claimed is:
 1. An information processing apparatus, comprising:one or more processors configured to: acquire an image andthree-dimensional point information on a three-dimensional point ofacquired surroundings of the image; assign an attribute to each regionin the image; set, to the three-dimensional point, the attributeassigned to the corresponding region in the image; and generate anobstacle map based on the three-dimensional point information and animpassableness probability corresponding to the attribute set to thethree-dimensional point, the obstacle map indicating a degree of beingpassable or impassable for each sectional region, wherein the one ormore processors are further configured to generate a first obstacle mapin which the impassableness probability is specified to the sectionalregion corresponding to the three-dimensional point to which theattribute indicating a solid object has been set, the first obstacle mapspecifying the impassableness probability, generate a second obstaclemap in which the impassableness probability is specified to thesectional region corresponding to the three-dimensional point to whichthe attribute indicative of being passable or impassable has been set,and generate the obstacle map in which the first obstacle map and thesecond obstacle map are integrated.
 2. The apparatus according to claim1, wherein the one or more processors are further configured to generatethe obstacle map corresponding to a bird's-eye view.
 3. The apparatusaccording to claim 1, wherein the attribute assigned by the one or moreprocessors indicates being passable, being impassable, or a solidobject.
 4. The apparatus according to claim 1, wherein, in the firstobstacle map generated by the one or more processors, a firstimpassableness probability, as the impassableness probability, isspecified to the sectional region corresponding to the three-dimensionalpoint to which the attribute indicating a solid object has been set, asecond impassableness probability that is lower than the firstimpassableness probability, as the impassableness probability, isspecified to another sectional region being in a range from a referenceposition to the sectional region to which the first impassablenessprobability has been specified, and an intermediate probability betweenthe first impassableness probability and the second impassablenessprobability, as the impassableness probability, is specified to anothersectional region being in a range farther away from the referenceposition than the sectional region to which the first impassablenessprobability has been specified.
 5. The apparatus according to claim 4,wherein, in the second obstacle map generated by the one or moreprocessors, the first impassableness probability is specified to thesectional region corresponding to the three-dimensional point to whichthe attribute indicative of being impassable has been set, the secondimpassableness probability is specified to the sectional regioncorresponding to the three-dimensional point to which the attributeindicative of being passable has been set, and the impassablenessprobability based on the sectional region to which the firstimpassableness probability or the second impassableness probability hasbeen specified is specified to the sectional region to which neither thefirst impassableness probability nor the second impassablenessprobability is specified.
 6. The apparatus according to claim 5, whereinthe one or more processors specify the impassableness probability, whichcorresponds to the attribute indicative of being impassable, to thesectional region corresponding to both the three-dimensional point towhich the attribute indicative of being passable has been set and thethree-dimensional point to which the attribute indicative of beingimpassable has been set.
 7. The apparatus according to claim 5, whereinthe one or more processors generate the obstacle map, in which the firstobstacle map and the second obstacle map are integrated, based onpriority of the impassableness probability indicating that the firstimpassableness probability is highest, the second impassablenessprobability is second highest, and the intermediate probability islowest, by specifying the impassableness probability of higher priorityto each sectional region included in the obstacle map out of theimpassableness probability specified to the sectional regioncorresponding to the first obstacle map and the impassablenessprobability specified to the sectional region corresponding to thesecond obstacle map.
 8. The apparatus according to claim 1, wherein theone or more processors display, on a display, a display screen includingthe obstacle map.
 9. The apparatus according to claim 1, furthercomprising a sensor configured to photograph the image and measure thethree-dimensional point information on the three-dimensional point ofthe acquired surroundings at time of photographing the image, whereinthe one or more processors are configured to acquire the image and thethree-dimensional point information from the sensor.
 10. The apparatusaccording to claim 9, wherein the one or more processors are furtherconfigured to measure the three-dimensional point information by using alaser.
 11. The apparatus according to claim 10, wherein the one or moreprocessors are further configured to measure the three-dimensional pointinformation based on an irradiation direction of the laser andinformation on reflected light of the laser at the three-dimensionalpoint.
 12. The apparatus according to claim 1, wherein the one or moreprocessors include a processor configured to control a power unit of amobile body based on information indicating the obstacle map.
 13. Acomputer program product including a non-transitory computer-readablemedium in which programmed instructions are stored, the programmedinstructions causing, when executed by a computer, the computer toperform a method comprising: acquiring an image and three-dimensionalpoint information on a three-dimensional point of acquired surroundingsof the image; assigning an attribute to each region in the image;setting, to the three-dimensional point, the attribute assigned to thecorresponding region in the image; and generating an obstacle map basedon the three-dimensional point information and an impassablenessprobability corresponding to the attribute set to the three-dimensionalpoint, the obstacle map indicating a degree of being passable orimpassable for each sectional region, wherein the method furthercomprises generating a first obstacle map in which the impassablenessprobability is specified to the sectional region corresponding to thethree-dimensional point to which the attribute indicating a solid objecthas been set, the first obstacle map specifying the impassablenessprobability, generating a second obstacle map in which theimpassableness probability is specified to the sectional regioncorresponding to the three-dimensional point to which the attributeindicative of being passable or impassable has been set, and generatingthe obstacle map in which the first obstacle map and the second obstaclemap are integrated.
 14. An information processing method, comprising:acquiring an image and three-dimensional point information on athree-dimensional point of acquired surroundings of the image; assigningan attribute to each region in the image; setting, to thethree-dimensional point, the attribute assigned to the correspondingregion in the image; and generating an obstacle map based on thethree-dimensional point information and an impassableness probabilitycorresponding to the attribute set to the three-dimensional point, theobstacle map indicating a degree of being passable or impassable foreach sectional region, wherein the method further comprises generating afirst obstacle map in which the impassableness probability is specifiedto the sectional region corresponding to the three-dimensional point towhich the attribute indicating a solid object has been set, the firstobstacle map specifying the impassableness probability, generating asecond obstacle map in which the impassableness probability is specifiedto the sectional region corresponding to the three-dimensional point towhich the attribute indicative of being passable or impassable has beenset, and generating the obstacle map in which the first obstacle map andthe second obstacle map are integrated.